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Dear  Susan, 

This  letter  is  the  fourth  quarterly  progress  report  for  grant  N00014-92-J- 
1945,  entitled  “Long  Term  Learning:  Integration  of  knowledge  acquisition 
and  knowledge  compilation.”  It  covers  the  period  April  1,  1993  to  June  30, 
1993. 

Two  goals  were  achieved  during  this  quarter.  First,  we  re-examined  one 
of  the  oldest  expert-novice  findings  in  the  literature:  when  solving  physics 
problems,  novices  do  write  equations  in  backwards  order,  starting  at  the 
sought  quantity  and  working  towards  the  given  quantities.  Experts  write 
equations  in  forwards  order.  This  was  first  noted  by  Simon  and  Simon,  and 
replicated  in  studies  by  Larkin  and  Sweller.  However,  it  is  an  odd  finding 
in  that  similar  strategy  shifts  that  could  occur  in  other  skills  (e.g.,  geometry 
theorem  proving  and  programming)  were  not  observed  to  occur.  In  those 
skills,  both  experts  and  novices  tended  to  use  a  variety  of  strategies,  but  the 
proportions  did  not  vary  with  levels  of  expertise.  We  begiin  by  asking  what 
made  physics  different  that  practice  should  cause  a  strategy  shift. 

We  began  by  seeing  what  it  would  take  to  make  Cascade  shift  from  backwards 
to  forwards  equation-writing.  We  designed  several  methods,  but  none  seems 
more  obviously  correct  than  the  others.  Moreover,  we  discovered  several 
different  ways  to  do  backwards  chaining,  and  we  were  not  sure  which  one 
was  used  by  novices.  We  re-read  Larkin’s  paper’s,  which  used  the  most 
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sophisticated  kind  of  physics,  but  the  papers  did  not  distinguish  among  the 
various  types  of  backwards  chaining.  Fortunately,  we  had  earlier  obtained 
Larkin’s  protocols  for  her  expert-novice  studies,  so  we  began  to  re-analyze 
them  in  order  to  determine  exactly  what  kind  of  strategy  the  novices  were 
employing. 

Much  to  our  surprise,  we  discovered  that  the  novices  were  not  consistently 
employing  any  of  the  backwards  chaining  strategies  we  knew  of.  They  would 
usually  forward  chain  up  to  the  point  where  they  had  drawn  a  force  diagram 
and  identihed  some  potentially  useful  equations.  What  they  did  next  seemed 
■to  vary  with  individuals  and  problems.  Sometimes  they  would  switch  to 
backwards  chaining  and  sometimes  they  would  continue  forwards  chaining. 
Sometimes  they  would  mix  the  two.  If  one  concentrates  only  on  the  written 
work,  then  there  does  not  appear  to  be  much  use  of  backwards  chaining 
among  the  novices.  If  a  subject  were  doing  backwards  chaining,  then  the 
first  equation  containing  the  problem’s  sought  should  be  in  position  1,  or 
perhaps  it  should  be  in  position  2  with  drawing  a  force-diagram  in  position 
1.  Instead,  the  average  position  was  7.5.  Subjects  typically  drew  a  force 
diagram  and  wrote  a  few  force  laws  before  writing  the  kinematics  equations 
that  contmned  the  sought  quantities. 


About  this  time,  I  gave  an  invited  presentation  at  the  AISB  conference  in 
Britain,  where  I  met  Tony  Priest,  who  has  been  working  on  and  off  for  years 
on  modelling  physics  problem  solving  strategies.  He  was  unhappy  with  the 
Larkin  study  because  it  had  been  done  with  only  a  small  number  of  subjects 
(9  novices;  11  experts)  and  an  even  smaller  number  of  physics  problems 
(2),  and  yet  this  sample  was  used  to  justify  a  general  assertion  applying  to 
all  novices  and  all  problems.  Priest  and  Lindsay  (1992,  British  Journal  of 
Psychology)  repeated  the  Larkin  study  with  more  subjects  (79)  and  more 
problems  (6).  They  found  no  evidence  of  an  expert-novice  shift.  At  all  levels 
of  expertise,  subjects  used  forwards  chaining  about  80%  of  the  time.  On  a 
hunch,  we  examined  all  the  mechanics  examples  of  the  Halliday  and  Resnick 
textbook,  and  they  also  solved  problems  in  forwards  order  about  80%  of  the 
time.  It  appears  that  the  original  Larkin  findings  were  a  sampling  artifact, 
and  that  there  is  no  expert-novice  difference  in  the  direction  in  which  the 
write  equations. 

However,  there  is  a  strategy  difference  between  experts  and  novices.  It  was 
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clear  in  Larkin’s  protocols  that  before  the  experts  write  their  first  equation, 
they  usually  have  a  plan  for  the  whole  solution.  Sometimes  they  plan  the 
solution  out  loud;  other  problems  are  so  simple  that  they  either  know  a  plan 
or  figure  it  out  so  rapidly  that  they  do  not  bother  to  talk  during  the  planning 
process.  Priest  and  Lindsay  (1992)  found  that  experts  could  produce  a  plan 
for  the  solution  whereas  novices  could  not  (as  did  Chi,  Feltovitch  and  Glaser, 
1981,  which  a  smaller  sample). 

These  findings,  as  well  as  several  others  in  the  expert-novice  literature,  led  us 
to  the  conclusion  that  the  chief  difference  in  the  problem  solving  strategies 
of  experts  and  novices  is  that  experts  can  plan  a  solution  in  their  heads  and 
novices  cannot.  This  casts  doubt  on  mechanisms  such  as  Soar’s  chunking  and 
act’s  production  compounding,  which  were  designed  to  cause  a  backwards- 
to-forwards  shift  in  strategy.  We  are  currently  of  the  opinion  that  no  such 
special  purpose  mechanisms  are  needed  to  account  for  the  expert’s  ability 
to  plan.  We  believe  that  novices  already  know  how  to  plan  with  equations 
because  they  learned  that  skill  during  high-school  algebra.  However,  they 
cannot  do  such  planning  in  physics  because  they  are  not  familiar  enough 
with  the  equations.  That  is,  they  might  be  able  to  tell  that  Nev/ton’s  law 
could  be  applied  to  a  certain  body,  but  they  would  not  be  able  to  determine 
which  quantities  would  be  involved  in  the  equation  without  first  writing  it 
down  and  doing  the  trigonometry.  The  expert  can  “just  see’’  which  quantities 
are  involved,  so  she  can  plan  out  the  effects  of  using  the  equation  without 
writing  it  down. 

We  plan  to  model  this  in  Cascade  by  having  the  productions  that  encode 
the  skill  of  planning  with  equations  require  that  the  strengths  of  the  be¬ 
liefs  encoding  the  equations  be  above  a  certain  threshold  before  planning 
can  successfully  occur.  This  is  a  crude  model  of  the  real  situation,  wherein 
familiarity  interacts  with  many  other  factors  to  determine  working  memory 
load  and  thus  the  capability  of  planning.  Modelling  the  real  situation  would 
require  a  finer-grained  model  of  memory  than  Cascade  has.  Our  proposed 
model  will  produce  the  observed  expert-novice  shift  and  allow  us  to  study  its 
interaction  with  other  forms  of  learning. 

The  second  achievement  of  this  quarter  was  isolation  of  some  expert  knowl¬ 
edge  for  system  definition.  When  we  modelled  the  solving  of  some  harder 
physics  problems,  we  noted  that  it  is  sometimes  not  obvious  how  to  choose 
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the  set  of  bodies  that  the  laws  would  apply  to  (the  system).  Experts  seem 
to  do  better  at  this  than  novices.  We  defined  and  tested  some  heuristics  for 
this  subskill,  but  we  are  not  yet  sure  how  the  expert  acquires  them. 


Best  regards, 


Kurt  VanLehn 

Assoc.  Prof,  of  Computer  Science 
LRDC  Senior  Scientist 

cc:  ONR  Resident  Representative,  Federal  Bldg,  Chicago  IL 
Director,  Naval  Research  Laboratory,  Code  2627 
Defense  Technical  Information  Center 
University  of  Pittsburgh  Contracts  Office 
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